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ABSTRACT 



This paper presents interface guidelines from research at 
the University of Pittsburgh (Pennsylvania) School of Information Sciences in 
developing a collaborative, problem-based learning environment for the World 
Wide Web. The lessons learned are based on student use and evaluation of 
three interface prototypes over the course of 3 years and spanning several 
domains: (1) CALE I, an X-Windows application that includes functionality to 

support synchronous as well as asynchronous collaborative problem-based 
learning; (2) CALE II, a Web-based version of CALE I; and (3) CoMMIT, a 
frames-based Web application. Insights into appropriate windowing strategies, 
choice of menu structure and presentation, menus as a group coordination 
mechanism, and group annotation mechanisms are discussed. Extensions to the 
interface based on these findings are discussed, and directions for future 
research are given. Three figures illustrate the CALE I, CALE II, and CoMMIT 
interfaces. Contains 15 references. (AEF) 
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Abstract: This paper presents interface guidelines from our research in developing a 
collaborative, problem-based learning environment for the WWW. The lessons learned are 
oo based on student use and evaluation of three interface prototypes over the course of three 

in years spanning several domains. Insight into appropriate windowing strategies, choice of 

o^ menu structure and presentation, menus as a group coordination mechanism, and group 

Tt annotation mechanisms are discussed. Extensions to our interface based on these findings 

Q are discussed and directions for future research are given. 



1.0 Introduction 



Over the past few years, the WWW has evolved from being a simple, document delivery mechanism to an 
increasingly complex, dynamic, and interactive environment. The ubiquitous nature of the WWW and the 
platform-independence that it provides to systems developers, provides opportunities to deploy systems that 
can be accessed virtually anywhere in the world, at any time, and on a wide variety of computers. The 
advances in telecommunication networks and groupware systems, coupled with the growing interest in 
distance education, provides a synergy which has led to an increased interest in using the Web as a formal 
educational delivery mechanism. 

The University of Pittsburgh School of Information Sciences has been actively involved in research to develop 
a computer-supported, collaborative learning environment [Mahling, et al. 1995]. Wliile our initial system 
was a UNIX/X-Windows-based learning environment, our more recent efforts are aimed at reaching a wider 
audience and supporting synchronous as well as asynchronous learning via the WWW. 

This paper presents our findings with respect to interface design strategies for computer-supported, 
collaborative, problem-based learning environments. The remainder of this paper is divided into three main 
parts. In section two, we introduce two educational scenarios for distance as well as collaborative learning 
within which our systems have been used and evaluated. In section three, we provide a chronological 
summary which highlights our research in developing collaborative learning environments to date. Section 
four discusses the collaborative learning interface requirements that we have discovered. 
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2.0 Computer Support for Collaborative Learning 

Developing effective instructional software for Computer-Supported Collaborative Learning (CSCL) demands 
that it be flexible enough to accommodate various patterns of use [Koschmann 1996]. In this section, we 
briefly review two of those CSCL scenarios, collaborative learning and asynchronous/synchronous problem- 
based learning, within which our collaborative learning environment has been tested. 
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2.1 Collaborative Learning 

The objective of collaborative learning is to encourage a group of students to work together to solve a problem. 
Collaborative learning strives to foster teamwork, individual accountability, prompt feedback, high self- 
expectations, and a respect for diversity among group members. Several studies have shown collaborative 
learning to be an effective model for education [McKeackie 1980; Kulik & Kulik 1979; Smith 1986]. Shared 
editing, synchronous and asynchronous work on a case, or navigating an information space together are some 
examples of opportunities where advanced computing technology can add to this pedagogical approach. 

2.2 Synchronous/Asynchronous Problem-Based Learning 

In recent years, problem-based learning (PBL) has received increased attention as a tool in medical 
curriculums and as the basis for designing new, innovative curricula in other fields as well. Medical schools 
have looked to PBL as a means to teach problem solving skills, to help students develop independent learning 
skills, and to create a bridge from lecture-based to more collaborative-based courses [Barrows 1994]. 

PBL helps students improve their reasoning skills by encouraging them to consolidate isolated facts into 
connected, conceptual clusters. PBL has been chiefly supported by conventional documents and “paper patient 
simulations”, though an increasing number of computer-supported environments are emerging [Grisson & 
Koschmann 1995; Mahling, et al. 1995; Hmelo, et al. 1995]. We believe that electronic information 
technology developed for the Web can truly unlock the potential of PBL for many learners in a variety of 
academic domains. Multimedia enables case materials to be represented very realistically. In addition, data 
systems minimize the bookkeeping chores found in PBL course administration. Also, the documentation 
created during the group’s approach to the problem can be automatically recorded. Advances in groupware 
research can be applied to provide computer support for cooperative, problem-based, distance learning. 

3.0 Mapping Stand-alone Applications to the Web 

Our research in computer support for collaborative learning began as a collaboration with the University of 
Pittsburgh School of Medicine. The collaboration was centered around how computers might help support the 
School of Medicine’s efforts in implementing a problem-based learning curriculum. Finding a more efficient 
way to deliver PBL cases to groups of students, as well as providing tools that support and facilitate 
collaboration among small groups of students, were among the chief concerns. 

3.1 CALE I: PBL for groups under UNIX 




Figure 1 CALE I Interface 
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CALE I (Fig. 1) is an X-Windows application and includes functionality to support synchronous as well as 
asynchronous collaborative problem-based learning. The system is a comprehensive, collaborative learning 
environment where students explore PEL cases on-line, take notes using a shared information space, and 
associate comments with case materials for future reference and learning by the group. CALE I was 
introduced as part of the University of Pittsburgh's Medial Decision Making course. 



3.2 CALE II: Porting an X- Windows Application to the WWW 




Figure 2 CALE II Interface 



The ability to reach a wider audience via the WWW resulted in CALE II, a web-based version of our 
collaborative learning environment (Fig. 2). CALE II was used by the University of Pittsburgh Pathology 
Department as part of their Integrated Life Science in Pathology course. The CALE II interface was restricted 
to a single window due to the limitations in the HTML standard at that time. The single window interface 
strategy placed a considerable cognitive load on students as they navigated and worked through PEL cases. 
The insight gained from student evaluations of the CALE II interface, coupled with advances in tools for Web 
application development, led us to develop our current web-based collaborative learning interface (CoMMIT) 
(Fig. 3). 

3.3 CoMMIT: A WWW PEL Interface Based on Frames 
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Figure 3 CoMMIT Interface 
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Commit is a frames-based web application and has been used in the University of Pittsburgh’s Department of 
Information Science and Telecommunications as part of an undergraduate course in Human-Computer 
Interaction. The remainder of this paper presents our findings resulting from empirical testing and evaluation 
of these three interfaces. 

4.0 PBL Interface Guidelines 



The look-and“feel of our collaborative learning environment has changed dramatically since its inception over 
four years ago. The evolution of these interfaces has revealed a number of interface design issues regarding 
appropriate windowing strategies, menu presentation strategies and structures, and annotation mechanisms 
that are conducive to computer support for collaborative, problem-based learning. 

4.1 Effects of Window Strategies on Case Navigation 

The memorization of isolated facts proves to be ineffective for complex problem-solving tasks [Spiro, et. al. 
1987]. PBL curriculums aim to overcome this on a case-level by requiring students to integrate information 
from several case documents to support their hypotheses or confirm their conclusions. System designers often 
employ a multiple-window interface strategy in situations where several information sources must be consulted 
simultaneously; however, the effects of single-vs.-multiple windows in computer-supported learning 
environments is still debated [Bly & Roesenberg 1986; Benshoof & Simon, 1993]. 

Each of our three prototypes employed a different windowing strategy to determine which is most effective. 
The CALE I system (Fig. 1) used an overlapping window strategy. Students were able to keep as many 
windows open as they liked; however, student response confirmed that this strategy often leads to feelings of 
being overwhelmed with “window-housekeeping chores” and not being able to spend enough time on the task. 
This finding is consistent with the findings in user-interface design research which points at the importance of 
letting the users focus on the domain tasks with minimal cognitive effort used for interface navigation [Card, 
Moran, & Newell 1983]. 

The web-based, CALE II interface (Fig. 2) employed a single window strategy (primarily because web 
development was not conducive to multi-window strategies at that time). A linear sequence of full-screen 
menu choices were presented to the students until the desired case material was eventually presented. Students 
again reported feelings of being “lost” and complained that they could not form an appropriate mental model 
of the case space or where they were within the case. The single-window model was clearly not appropriate. 
It is interesting to note that neither the total flexibility of multiple overlapping windows, nor the rigidity of 
single window task focus were appropriate for the learners. 

The Commit interface displays both the main and corresponding secondary menus at all times to facilitate 
students’ navigation through a case. Student-requested case documents are presented in a separate tiled 
window. A group Notepad resides in an accompanying, floating window to support the need to organize group 
thoughts and ideas during case exploration. Overall, students have responded positively to a tiled-window 
strategy coupled with a floating Notepad window, yet simultaneous presentation of multiple documents is still 
a problem. We are currently extending the functionality of the CoMMIT interface to employ a combined tiled 
and overlapping windows approach to allow for the viewing of multiple case documents simultaneously. 

4.2 Menu Structure and Presentation 

PBL presents a challenge for the system designer to determine an effective way for structuring and providing 
access to case documents such that the system is conducive to case exploration. Students follow an iterative 
cycle of requesting information, analyzing and integrating this information with what is already known, and 
determining whether the case can be solved or if the cycle should be repeated. Supporting this high level of 
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information requests requires the thoughtful choice from a host of menu structuring strategies. A 
comprehensive taxonomy of menu strategies has been suggested by Schneiderman [Schneiderman 1992]. 

When users have a large number of selections from which to choose, menus organized by categories are an 
effective strategy [Norman 1991]. Students using our system find a two-tiered menu structure with domain- 
specific categories on the Main Menu and corresponding case materials on a corresponding second-level 
menu. This strategy allows PBL case authors to help shape the students mental model of the domain and use 
menu terminology that is familiar to the students. Students expressed that this two-tiered menu structure 
works well, but only if the menus are visible at all times. 

The single interface of CALE II required us to modify the presentation of our two-tiered menu. In CALE II, 
students would first be presented with a list of Main Menu options. After selecting an item from the Main 
Menu, the system next presented the second-level menu with options corresponding to the Main Menu choice. 
Choosing one of the secondary menu options resulted in a case material being presented. After students were 
finished viewing a case material, the system would return them to the Main Menu and the cycle would begin 
again. This linear presentation of two-tiered menus resulted in a substantial cognitive overload for students. 
Students expressed feelings of “getting lost” in the menu structures and felt that it was difficult to form an 
appropriate mental model of the case document space. 

We have found that the availability of the menu at all times is critical for collaborative, PBL environments. In 
the Commit interface, students can see the Main and Second-level Menus at all times - each menu resides in 
a separate tiled window. Choosing an item from the Main Menu updates the Second-Level menu. Selecting 
an item from the Second-Level Menu presents the selected document in the case material window. Using this 
menu model allows students to see how they got to a particular case material and reminds them of the 
document categories from which they can choose. 

An interesting student behavior that we observed is that students rely on the menu not only as case material 
selection mechanism, but also as a mechanism for coordinating group activities. Student evaluations of all 
three interfaces suggest that the menu structure should include status information such as which options were 
attempted by the group, whether or not the request was successful or not, and if not, how many times had the 
option been tried. We look forward to incorporating these suggestions into our next version of CoMMIT and 
assessing its utility in facilitating group problem-solving. 

4.3 Context-sensitive classification of Case Annotations 

Students in paper-based PBL often use a physical blackboard divided into four columns: Facts, Hypotheses, 
Learning Issues (to do*s) and Actions to help organize the group’s thoughts and ideas during case exploration 
[Meyers, et al.l990]. In the paper-based PBL environment, one student in the group acts as a scribe to record 
and update the information in these four categories on the blackboard as the group proceeds through the case. 
To support this requirement in our system, we developed a shared information space called the NotePad that 
follows the blackboard metaphor. During case exploration, students switch to the Notepad Window to record 
information in any of the four categories as the need arises. The system records the name, time, and date of 
student annotations and orders those annotations from most-to-least recent. 

We found that the students perception of the blackboard metaphor changed when the blackboard was 
implemented electronically. Students suggested that while the blackboard metaphor provided some computer- 
support for the group’s information needs, they preferred to enter this information directly with the case 
material rather than using a separate Notepad window. We found that students often used our Margin Note 
feature (originally intended for making only general comments “in the margins” of displayed case materials) 
in lieu of the Notepad when entering information about facts/hypotheses, etc. This practice was consistent 
across all groups in all domains that used our system. 

An analysis of this phenomenon led us to conclude that supporting the Margin Note approach to group case 
annotation has several merits: 
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students remain focused on the task of annotating rather than concerning themselves with window 
management tasks of switching back and forth between the case material and Notepad windows, 
annotations used with the Margin Note feature provided a richer context within which to understand 
student annotations thus students were more likely to annotate for both themselves and for the benefit of 
the group. 

because the annotations were more contextually dependent, facilitators could more accurately assess the 
breadth and depth of the students’ knowledge and reasoning which is a fundamental principle of PBL 
[Koschmann 1996]. 

annotations made with the case material can be classified by the students at entry and automatically 
indexed in the NotePad such that group activities can be viewed at a glance. In this way, the NotePad can 
serve as a point of departure for future collaborative sessions on the case. 

5.0 Summary of Lessons Learned 

Our initial efforts at supporting collaborative learning in a problem-based learning environment were 
concerned with providing a shell within which PBL cases could be delivered to groups of students. Although 
our initial systems did provide computer support for collaborative problem-based learning, our experiences in 
implementing three different interfaces helped us uncover more subtle interface requirements for this type of 
learning environment. Specifically: 

students prefer a semi -structured window management strategy over a totally unstructured or totally rigid 
window management scheme, 

a two-tiered, hierarchical menu structure is effective for students to form and maintain a mental model of 
the case document space but only if those menus are displayed together and at all times, 
the blackboard metaphor is only partially effective in our computer-supported PBL environment. Students 
prefer to organize hypotheses, facts, and action items at the point of entry (with the case materials 
themselves) rather than using a blackboard metaphor, 

a sorted, centralized compilation of student case material annotations (done automatically by the system) 
provides a high-level perspective of group activities. These centralized compilations can permit both 
students and facilitators to more accurately audit the group’s knowledge and problem-solving processes. 
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